Real Time Data Replication

ABSTRACT

A combination of synchronous and asynchronous replication of data is used to replicate a local database to a replicated database. The typical tradeoff between synchronous and asynchronous replication is optimized by using hybrid replication, which is to use synchronous replication for inserting new data and asynchronous replication for updating existing data. The combined use of synchronous and asynchronous in this manner provides an efficient replicated database where the replicated database can tolerate some delay in data updates but requires no data loss of new data.

BACKGROUND

1. Technical Field

This disclosure generally relates to replication of computer resources,and more specifically relates to data replication which combinessynchronous and asynchronous methods to provide fast and efficient realtime data replication.

2. Background Art

There is an ever increasing need for reliable high speed data processingsystems. These data processing systems require data repositories thatcan provide critical data at any moment in time in the face ofinterruptions caused by hardware failure, software failure, naturaldisaster, etc. To achieve the necessary data availability, many datasystems utilize remote data repositories to backup and store criticaldata. A common means of fast data backup to the remote data repositoriesis by database replication.

With database replication, the database management system can makeinformed decisions on whether to write data to multiple local storagedevices or to a local storage device and a remote storage device, butsuch synchronization comes at a significant performance penalty. Thetechnique of replicating data to multiple storage devices simultaneouslyis also known as mirroring. Data replication is typically eithersynchronous or asynchronous. In synchronous data replication, thereplicated database is kept updated with current information in theprimary database. In this manner, critical data can be accessible at alltimes. Ensuring transaction and record consistency often results in datatransmission latency when a large number of data transmissions to remotesites are necessary with each database update. Consequently, applicationperformance may be slowed to unacceptable levels.

Database replication systems also use asynchronous replication. Inasynchronous systems, the data in the replicated database is updated inbatches or at specified intervals. While asynchronous systems use lesssystem resources, they do not guarantee there will be no data loss onthe replicated database if a failure of the primary database occurs.Therefore, applications that require no loss of data at the replicateddatabase often cannot employ asynchronous replication and thus rely onslower synchronous replication techniques.

While asynchronous systems are faster and use less system resources,they do not guarantee there will be no data loss on the replicateddatabase. Synchronous replication systems can guarantee no loss of databut are slower and have a higher cost to system resources. Thus there isa tradeoff between using asynchronous and synchronous data replication.

BRIEF SUMMARY

The disclosure and claims herein are directed to data replication usinga combination of synchronous and asynchronous replication of data from alocal database to a replicated database. The typical tradeoff betweensynchronous and asynchronous replication is optimized by using hybridreplication, which is to use synchronous replication for inserting newdata and asynchronous replication for updating existing data. Thecombined use of synchronous and asynchronous in this manner provides anefficient replicated database where the replicated database can toleratesome delay in data updates but requires no loss of new data.

The foregoing and other features and advantages will be apparent fromthe following more particular description, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The disclosure will be described in conjunction with the appendeddrawings, where like designations denote like elements, and:

FIG. 1 is a block diagram of a computer system with a hybrid replicationmechanism to replicate a database as described herein;

FIG. 2 is a block diagram of a replication system according to the priorart;

FIG. 3 illustrates a block diagram of a replication system using hybriddata replication; and

FIG. 4 is a method flow diagram for replicating a database using hybriddata replication as described herein.

DETAILED DESCRIPTION

The disclosure and claims herein are directed to data replication usinga combination of synchronous and asynchronous replication of data from alocal database to a replicated database. The typical tradeoff betweensynchronous and asynchronous replication is optimized by using hybridreplication, which is to use synchronous replication for inserting newdata and asynchronous replication for updating existing data. Thecombined use of synchronous and asynchronous in this manner provides anefficient replicated database where the replicated database can toleratesome delay in data updates but requires no data loss of new data.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CD-ROM), an optical storage device, a magnetic storage device, or anysuitable combination of the foregoing. In the context of this document,a computer readable storage medium may be any tangible medium that cancontain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device. A computer readablesignal medium may include a propagated data signal with computerreadable program code embodied therein, for example, in baseband or aspart of a carrier wave. Such a propagated signal may take any of avariety of forms, including, but not limited to, electro-magnetic,optical, or any suitable combination thereof. A computer readable signalmedium may be any computer readable medium that is not a computerreadable storage medium and that can communicate, propagate, ortransport a program for use by or in connection with an instructionexecution system, apparatus, or device. Program code embodied on acomputer readable medium may be transmitted using any appropriatemedium, including but not limited to wireless, wireline, optical fibercable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider). Aspects of the present invention are described below withreference to flowchart illustrations and/or block diagrams of methods,apparatus (systems) and computer program products according toembodiments of the invention. It will be understood that each block ofthe flowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks. These computer program instructions may also be stored in acomputer readable medium that can direct a computer, other programmabledata processing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks. The computer program instructions may also beloaded onto a computer, other programmable data processing apparatus, orother devices to cause a series of operational steps to be performed onthe computer, other programmable apparatus or other devices to produce acomputer implemented process such that the instructions which execute onthe computer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Referring to FIG. 1, a computer system 100 is one suitableimplementation of a computer system that includes a hybrid replicationmechanism as described herein. Computer system 100 is an InternationalBusiness Machines Corporation (IBM) iSeries computer system. However,those skilled in the art will appreciate that the disclosure hereinapplies equally to any computer system, regardless of whether thecomputer system is a complicated multi-user computing apparatus, asingle user workstation, or an embedded control system. As shown in FIG.1, computer system 100 comprises one or more processors 110, a mainmemory 120, a mass storage interface 130, a display interface 140, and anetwork interface 150. These system components are interconnectedthrough the use of a system bus 160. Mass storage interface 130 is usedto connect mass storage devices with a computer readable medium, such asdirect access storage devices 155, to computer system 100. One specifictype of direct access storage device 155 is a readable and writableCD-RW drive, which may store data to and read data from a CD-RW 195.

Main memory 120 preferably contains an operating system 121. Operatingsystem 121 is a multitasking operating system known in the industry asi5/OS; however, those skilled in the art will appreciate that the spiritand scope of this disclosure is not limited to any one operating system.The memory further includes a database manager 122 that includes ahybrid replication mechanism 123. The memory further contains at least aportion of a local database 124. The local database 124 may also bestored in the direct access storage device 155 and portions of thedatabase may be loaded into memory 120 as needed. These entities inmemory are described further below.

Computer system 100 utilizes well known virtual addressing mechanismsthat allow the programs of computer system 100 to behave as if they onlyhave access to a large, single storage entity instead of access tomultiple, smaller storage entities such as main memory 120 and DASDdevice 155. Therefore, while operating system 121, database manager 122,hybrid replication mechanism 123, and local database 124 are shown toreside in main memory 120, those skilled in the art will recognize thatthese items are not necessarily all completely contained in main memory120 at the same time. It should also be noted that the term “memory” isused herein generically to refer to the entire virtual memory ofcomputer system 100, and may include the virtual memory of othercomputer systems coupled to computer system 100.

Processor 110 may be constructed from one or more microprocessors and/orintegrated circuits. Processor 110 executes program instructions storedin main memory 120. Main memory 120 stores programs and data thatprocessor 110 may access. When computer system 100 starts up, processor110 initially executes the program instructions that make up operatingsystem 121 and later executes the program instructions that make up thedatabase manager 122 and hybrid replication mechanism 123.

Although computer system 100 is shown to contain only a single processorand a single system bus, those skilled in the art will appreciate that avirtualization manager may be practiced using a computer system that hasmultiple processors and/or multiple buses. In addition, the interfacesthat are used preferably each include separate, fully programmedmicroprocessors that are used to off-load compute-intensive processingfrom processor 110. However, those skilled in the art will appreciatethat these functions may be performed using I/O adapters as well.

Display interface 140 is used to directly connect one or more displays165 to computer system 100. These displays 165, which may benon-intelligent (i.e., dumb) terminals or fully programmableworkstations, are used to provide system administrators and users theability to communicate with computer system 100. Note, however, thatwhile display interface 140 is provided to support communication withone or more displays 165, computer system 100 does not necessarilyrequire a display 165, because all needed interaction with users andother processes may occur via network interface 150.

Network interface 150 is used to connect computer system 100 to othercomputer systems or workstations 175 via network 170. These othercomputer systems 175 preferably include one or more replicated databases125. Network interface 150 broadly represents any suitable way tointerconnect electronic devices, regardless of whether the network 170comprises present-day analog and/or digital techniques or via somenetworking mechanism of the future. In addition, many different networkprotocols can be used to implement a network. These protocols arespecialized computer programs that allow computers to communicate acrossa network. TCP/IP (Transmission Control Protocol/Internet Protocol) isan example of a suitable network protocol.

FIG. 2 illustrates a block diagram of a computer system 200 with datareplication according to the prior art. The system 200 includes twocomputers 210 connected with a network 212. Each computer 210 includes areplication mechanism 214 that together replicate a local database 216to a replicated database 218. The local database 216 may be replicatedto the replicated database 218 using synchronous 220 or asynchronous 222replication. The prior art system 200 employs synchronous 220 orasynchronous 222 replication depending on the needs of the applicationand the constraints of the overall system hardware. Note, however knownprior art systems (such as 200 in FIG. 2) use one or the other,synchronous or asynchronous replication. Prior art systems do not useboth synchronous and asynchronous replication in the manner describedherein.

FIG. 3 illustrates a block diagram of a computer system 300 using hybriddata replication as described and claimed herein. Hybrid datareplication uses a combination of synchronous and asynchronousreplication to provide an efficient replicated database where thereplicated database can tolerate some delay in data updates but requiresno data loss of new data. The system 300 includes two computers that mayinclude the features of computer 100 shown in FIG. 1 and correspond tocomputers 100 and 175 in FIG. 1. The two computers 100, 175 areconnected with a network 170. The computers 100, 175 include a hybridreplication mechanism 123, 312 that together replicate a local database124 to a replicated database 125. The local database 124 is replicatedto the replicated database 125 using hybrid replication 310. As usedherein hybrid replication means synchronous replication is used fortransaction that add new data to a database memory (such as INSERT 312operations), and asynchronous replication is used for transactions thatupdate or replace existing data in a database memory (such as UPDATE 314database transactions). As used herein, an INSERT operation is adatabase operation that inserts a new database record into a database.Similarly, an UPDATE transaction is a database operation that changes orupdates an existing record in a database. The claimed inventionexpressly extends to other data memory operations that have similarcharacteristics of memory UPDATE and INSERT as described above.

We will now consider an example of using hybrid database replication asdescribed above. The application for this example is a military radarsystem that sweeps an enemy location every 30 ms for enemy targets. Inthis situation, a primary concern is that all new threats show up in theradar display without risk of loss of these threats. Enemy threatsdetected by the radar system are stored in a local database. A databasereplication system is employed to replicate the database holding thethreat information to insure against a loss of the primary database. Ifthere is a system failure, the replicated database must contain alltargets on the primary system. Position of the threats will be updatedon the next sweep, so the most up to date location data is not critical.

Continuing with the radar example, we assume at time t=0 the radarsystem reads threats A and B. These threats and their coordinates aresynchronously replicated on the backup database with an INSERTtransaction. At the 30 ms sweep, the radar detects a new location forthe targets A and B. The new coordinates are replicated asynchronouslywith an UPDATE transaction. At the 60 ms sweep, the radar detects a newlocation for the targets A and B and a new target C. The new coordinatesfor A and B are replicated asynchronously with an UPDATE transaction,and the coordinates for the new target C is replicated synchronously. Attime=69 ms, we assume a failure on the primary database occurs. As aresult, the data on the replication system is read. The backup dataholds data for all the targets, but the coordinates for A and B may bestale due to the asynchronous transaction not being completed before thefailure. This stale data is not a critical problem but it is importantthat all the targets are accounted for on the backup database. At thenext radar sweep, the new coordinates for A, B, and C are updated to thereplicated database with an UPDATE transaction. The above description issummarized in the table below.

Time Radar Detection Replication Transaction  0 ms A, B Insert Tx - A, BSynchronously 30 ms A, B Update Tx - A, B Asynchronously 60 ms A, B, CUpdate Tx - A, B Asynchronously Insert Tx - C Synchronously 69 msFailure Use Backup system 90 ms A, B, C Update Tx - A, B, CAsynchronously

FIG. 4 shows a method 400 for to data replication using a combination ofsynchronous and asynchronous replication of data from a local databaseto a replicated database. The steps in method 400 are preferablyperformed by the hybrid replication mechanism 123 (FIGS. 1 and 3) inconjunction with the hybrid replication mechanism 312 (FIG. 3) toprocess a database transaction to replicated it to a replicateddatabase. First, determine if the transaction is complete (step 410). Ifthe transaction is complete (step 410=yes) then the method is done. Ifthe transaction is not complete (step 410=no) then determine if thetransaction is an insert (step 420). If the transaction is not an insert(step 420=no) then update the transaction data locally and replicate tothe replicated database using asynchronous replication (step 430). Next,mark the transaction as complete (step 440) and go to step 410. If thetransaction is an insert (step 420=yes) then insert the transaction datainto the local database and replicate the data to the replicateddatabase using synchronous replication (step 450). Next determine if thereplication is successful (step 460). If the synchronous replication wassuccessful (step 460=yes) then mark the transaction as complete (step440) and go to step 410. If the synchronous replication was notsuccessful (step 460=no) then check the number of times the replicationwas attempted (step 470). If the retry is N times (step 470=yes) thenreport an error (step 480). If the retry has not been N times (step470=no) then retry the replication (step 490) and return to step 460.Note that the number N in FIG. 4 is an integer threshold that determineshow many times replication will be attempted before reporting an error.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

As described above, a database manager utilizes a hybrid replicationmechanism that combines synchronous and asynchronous replication of datafrom a local database to a replicated database. The combined use ofsynchronous and asynchronous in this manner provides an efficientreplicated database where the replicated database can tolerate somedelay in data updates but requires no data loss of new data.

One skilled in the art will appreciate that many variations are possiblewithin the scope of the claims. Thus, while the disclosure isparticularly shown and described above, it will be understood by thoseskilled in the art that these and other changes in form and details maybe made therein without departing from the spirit and scope of theclaims.

1. A computer-implemented method for hybrid replication of a database,the method comprising the steps of: (A) determining if a databasetransaction is a transaction that adds new data to the database orupdates data on the database; (B) for a database transaction thatupdates data in the database, updating the data locally and replicatingthe data to the replicated database asynchronously; (C) for a databasetransaction that adds new data, adding the new data locally andreplicating the data to the replicated database synchronously; and (E)wherein the method steps are implemented in a computer software programstored in computer memory and executed by a computer processor.
 2. Themethod of claim 1 wherein the transaction to add new data to thereplicated database is a database INSERT operation and the asynchronousreplication transaction to update data in the replicated database is adatabase UPDATE operation.
 3. The method of claim 1 wherein thereplicated database is associated with another computer system locatedremotely from the at least one processor.
 4. The method of claim 1wherein the replicated database holds radar information that is updatedusing hybrid replication.
 5. The method of claim 1 wherein the step ofreplicating the data synchronously determines if the replication wassuccessful, and if not successful, then retrying the replication “N”times.
 6. An article of manufacture comprising a replication mechanismstored on a computer recordable media that when executed by a computerprocessor performs the steps of: (A) determining if a databasetransaction is a transaction that adds new data to the database orupdates data on the database; (B) for a database transaction thatupdates data in the database, updating the data locally and replicatingthe data to the replicated database asynchronously; (C) for a databasetransaction that adds new data, adding the new data locally andreplicating the data to the replicated database synchronously; and (D)wherein the method steps are implemented in a computer software programstored in computer memory and executed by a computer processor.
 7. Thearticle of manufacture of claim 6 wherein the transaction to add newdata to the replicated database is a database INSERT operation and theasynchronous replication transaction to update data in the replicateddatabase is a database UPDATE operation.
 8. The article of manufactureof claim 6 wherein the replicated database is associated with anothercomputer system located remotely from the at least one processor.
 9. Thearticle of manufacture of claim 6 wherein the replicated database holdsradar information that is updated using hybrid replication.
 10. Thearticle of manufacture of claim 6 wherein the step of replicating thedata synchronously determines if the replication was successful, and ifnot successful, then retrying the replication “N” times.